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Abstract. Maintaining optical alignment between stations of a free-space optical link 
requires a persistent beam tracking operation. This is achieved using a position-sensitive 
photodetector at each station which measures the azimuth and elevation of tracking error. 
A pointing assembly adjusts the heading of transceivers according to measurement of the 
tracking error. The measurement at each receiver also depends on the pointing error of 
the opposite transmitter, therefore a cooperative beam tracking system can be regarded as 
two dynamically coupled subsystems. We developed a stochastic model for a cooperative 
beam tracking system to get insight into solution of an associated optimal control problem 
with goal of maximizing the flow of optical power between the stations. 



1. Introduction 

In free-space optical communication using narrow laser beams, it is required to maintain 
the alignment of transmitter and receiver stations in spite of their relative motion. This 
relative motion might be caused by the mobile nature of the stations, mechanical vibra- 
tion, or accidental shocks. Prior to data transmission begins, coarse alignment is achieved 
through two operations: pointing and spatial acquisition. Pointing is the act of aiming 
the transmitted beam toward the receiver within an acceptable accuracy. The purpose of 
spatial acquisition is to detect the transmitter's beam and align the normal vector to the 
receiving optical device with the direction of the impinging optical field. Following a coarse 
alignment accomplished in the acquisition phase, data transmission is established and si- 
multaneously the operation of cooperative beam tracking is performed. This fine alignment 
operation is intended to precisely compensate for persistent relative motion of the stations. 
For detailed description of these three operations we refer the reader to [Hl^. In this study 
we are concerned about cooperative beam tracking. 

In a cooperative beam tracking system, the stations continuously track the arrival di- 
rection of the incident beam and transmit their beam back in that direction. A position- 
sensitive photodetector (e.g. quadrant detector) in association with a focusing lens is em- 
ployed at each station to measure the azimuth and elevation components of the error vector. 
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The error vector is the displacement of the beam's arrival direction with respect to the di- 
rection normal to the receiving aperture. A servo-driven pointing assembly adjusts the 
heading of the transmitting optical device (in azimuth and elevation directions) accord- 
ing to the measured error vector. Normally, a single pointing assembly is used to control 
the direction of both receiving and transmitting optics, which are installed on the same 
platform. 

Cooperative tracking systems were already studied by other researchers based on deter- 
ministic models [lll3l|4]. These models describe the position-sensitive photodetector by a 
deterministic input-output relationship and employ deterministic functions to characterize 
the relative motion of the stations. Using such a deterministic model, under a proportional 
control law, Wei and Gagliardi [3J evaluated the steady-state performance of the system 
and Marola et al. studied stability properties 

For most applications, a stochastic model may be more appropriate for relative motion 
of the stations. Moreover, the optical sensing devices are usually described by stochastic 
models [H [5] . This leads us to suggest that a stochastic approach is more likely to lead 
to rigorous analysis and design of cooperative tracking systems. In this paper, we develop 
a stochastic model for cooperative tracking systems and employ the model to analyze the 
system and study an associated optimal control problem. Maximizing the flow of the optical 
power between the stations shall be our criterion for optimality. 

2. System Description 

We consider an optical transceiver comprised of a lens, a position-sensitive photodetector, 
and a narrow laser source, all installed on a rigid platform. The photodetector surface is 
perpendicular to the lens axis and its center is placed at the focus of the lens. The axes of 
the lens and the laser source are parallel to the transceiver axis. The azimuth and elevation 
of the transceiver axis can be controlled by means of a servo-driven pointing assembly. 
A two-way optical link employs two transceivers of this type in such a manner that each 
transceiver transmits its optical beam toward the opposite station and receives the optical 
beam from the opposite side. The optical beams are used for two purposes: as a carrier 
of information and as a beacon assisting the opposite station in its tracking operation. We 
assume that the stations are subjected to relative motion. 

In what follows, we distinguish the stations by superscripts a and b or i = a,b when 
referring to both stations. The dependence on time will be shown by subscript t. The two- 
dimensional vector 91 denotes the azimuth and elevation angles of the transceiver axis i with 
respect to some fixed coordinate system. Similarly, (pt denotes the azimuth and elevation 
angles of the line-of-sight (LOS) of the stations with respect to same coordinate system. 
We define the tracking error of station i as ipl = 91 — (j)t which is equivalent to the pointing 
error for the opposite station. 

We shall assume that the transmitted optical fields impinge the receivers along the line- 
of-sight of the stations, regardless of the pointing error of the transmitters. This implies that 
the received optical field at station i strikes the receiving lens along the error vector ipl with 
respect to the axis of the lens. The validity of this assumption is clear for a spherical optical 
field. For a Gaussian beam [6j, which is the model used in this study, we can show that 
the angle between LOS and the beam arrival direction at the receiver depends on the third 
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power of the pointing error at the transmitter. Because the pointing error is maintained 
small by means of feedback control, its third power can be neglected at least for a linear 
model, which justifies our assumption. 

In contrast to the beam arrival direction, the intensity of the received optical field sub- 
stantially depends on the transmitter's pointing error. Assume that station b transmits a 
circular symmetric Gaussian beam with divergence angle 2^ toward station a. Then, due 
to the pointing error ip'l at station b, the instantaneous optical power received by station a 
at time t is reduced by a factor of exp (— 2||'i/'^|p/^^) . This attenuation factor is obtained 
based on two assumption: first, the radius of the optical beam is much larger than the 
receiver aperture, and second, the distance between the stations is short enough to allow 
ignoring the propagation delay. 

The image of the received optical field over the surface of the photodetector is a spot of 
light with a bell-shaped intensity profile centered at yl = fcipl, where fc is the focal length 
of the lens [1]. Let Q (r) be the intensity pattern of the spot of light, where r is the position 
vector of a point on the surface of the photodetector. Denote by > the total optical 
power received by station a at time t in the absence of pointing error ip^. Then the optical 
intensity over the surface of photodetector a is given by 

(r) = P,"exp {-2U1\\y^P^) n{r- (1) 

A similar expression can be obtained for (r) by flipping a and 6 in ([1]). Note that the 
displacement of the spot of light depends only on the error vector of the same station, while 
the total received power depends on the error vector of the opposite station. 

Since ipl depends linearly on the displacement of the spot of light, it can be estimated 
from the output of the position-sensitive photodetector. This estimate is provided to a 
controller which applies proper control signals to the pointing assembly in order to drive 
the error vector to zero. Note that ipf = il^i = is the "unattainable" goal of a cooperative 
tracking system. Under a realistic condition, the objective of the system is to maintain 
and \\ipt\\ as small as possible. Since the axis of the laser source is parallel to the transceiver's 
axis, 1^1 = implies that the station i transmits its optical beam along LOS, which leads 
to maximum flow of the optical power between the stations. 

From the above description, we find out that a cooperative tracking system consists 
of two dynamical subsystems coupled via their measurement. The subsystems operate 
cooperatively in the sense that a small pointing error at one station assists the other station 
by increasing its received optical power which leads to more accurate estimation of its error 
vector. 

3. The Model 

The model we use in this study is a two-station extension of the single-station model 
in [7j. We refer the reader to that article for detailed description and justification of the 
model. Without loss of generality, we assume the stations are identical. 

The pointing assembly is an electro-mechanical system with the input vector € and 
the output vector 0j G R^. The input and output vectors are two-dimensional correspond- 
ing to the azimuth and elevation angles. We model this system by the linear stochastic 
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differential equation 

dxf = A\xfdt + n^dt + D^^dwl'' 



31 = CI 



(2) 



where xF^^ G M"?' is the state vector, ^ 0} is a mp-dimensional standard Wiener 

process, and , , L'f , and Cf are uniformly bounded matrices with proper dimensions. 
We assume that ^ 0} and {w\^ ^ 0} are mutually independent. 

Using a linear model for the pointing assembly is justified by the fact that the system 
operates over small angles during the fine control regime. In applications like intersatel- 
lite communication, the relative motion consists of a large, deterministic component and a 
small, stochastic term. Accordingly, the control law consists of a deterministic, open-loop, 
coarse control and a small, closed-loop, fine control. In this case, the nonlinear state equa- 
tions describing the system is linearized around the deterministic nominal trajectory, which 
results in the time-varying model ([2]) for the fine control regime. 

We model (^t by a Gauss-Markov stochastic process described by the state-space equations 

dxi = Afxfdt + Dfdwf 
<Pt = Cfxt 

with state vector S M"'*, m^-dimensional standard Wiener process {wf,t ^ 0}, and 
uniformly bounded matrices A^, Df, and Cf with proper dimensions. 

The position vector yl = fcipl is a linear function of x^'^ and xf, so we can combine 
equations ([2]) and ([3]) and write them in the compact form 

dxl = Atx\dt + Btu\dt + Dtdw\ 

vi = Ctx\ 



(4) 



with state vector x\ € M" and m-dimensional standard Wiener process {w\,t ^ 0}, where n = 
np + rid and m = nip + m^. The initial state Xq is assumed to be a Gaussian vector with 
mean Xq and covariance matrix Sg and independent of {wf,t ^ 0} and {wf,t ^ 0}. 

We approximate the bell-shaped intensity profile (r) in ([T]) by a Gaussian function. 
Then, in terms of the state vectors and x^, the optical intensity /f (r) can be expressed as 

If (r) = Pf exp [-p\\Ctx',f) It (r, x?) (5) 

where p = 2/(V'/c)^ and 7^ (r, x) is defined as 

7t (r,x) = (27r)-^ (det i?*)"^/' exp j-^ (r - Ctxf (Rt)-^ (r - Ctx)^ (6) 

Here, Rt = RJ is a 2 x 2 positive-definite matrix describing the shape of the pattern. For 
a circular symmetric pattern with constant radius ^> > we have Rt = qI2x2- A similar 
expression can be derived for (r) by exchanging a and b. We remind that in ([5]) the 
propagation delay is neglected. 

We allow P^^ and Pj^ to be nonnegative stochastic processes with piecewise continuous 
sample paths and nonzero expectations to model the random optical fade caused by atmo- 
spheric turbulence and aerosols and the information-bearing signals modulating the optical 
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beams. Further, we assume that {Pt,t ^ 0} and {Pt,t ^ 0} are mutually independent and 
independent of Xq and {wl,t ^0}, i = a,b. 

The position-sensitive photodetector is a photoelectron converter whose surface is par- 
titioned into small regions. The output of each region counts the number of converted 
electrons regardless of their location on the region. The photoelectron conversion rate de- 
pends linearly on the optical power absorbed by the region. Generally, a photoelectron 
converter is modeled by a Poisson process with a rate proportional to the impinging optical 
power [HIS]. In the present case, where the optical power is a stochastic field, the output 
of each region shall be modeled by a conditionally (doubly stochastic) Poisson process. 

Many practical beam tracking systems employ a quadrant detector, a photodetector with 
a four-region partition, as their optical sensing device. However, the low spatial resolution 
of the quadrant detector can be improved using a finer partition. For instance, the authors 
of [8] describe a beam tracking system utilizing a photodetector with 512 x 512 pixels. In 
this study, following [7], we use an infinite resolution model for the photodetector. This 
idealized model provides a reasonable approximation for high resolution photodetectors. 
We also make another ideal assumption that the surface of the photodetector is infinitely 
large [7]. This assumption is justified when the photodetector area is significantly larger 
than the size and the displacement of the spot of light. We believe that the control law 
obtained from this idealized model provides a useful point of departure for practical designs, 
even for low resolution or finite area photodetectors. 

We use a doubly stochastic space-time Poisson process to describe the output of an 
infinite resolution photodetector [7]. The rate of this process is assumed to be proportional 
to II (r). From ([5]), the stochastic rate associated to photodetector i can be expressed as 

Xt{r,xi,4) = 4-ft{r,xi) (7) 

where /if and //^ are defined as 

/xf = z.»exp(-p||Ci4f) 
^,1 = u^eM-p\\Ctxt\\') 

Here, vl is defined as vl = i^Pl, where r/ > is the sensitivity of the photodetectors and is 
assumed to be a constant. Note that ul inherits the statistical properties of P^. 

The space-time Poisson process, defined over [0, oo) x M^, characterizes the occurrence of 
discrete events (e.g. release of a single electron) with a temporal component t G [0, oo) and 
a spatial component r G M^. Let T and S be Borel sets in [0, oo) and respectively and 
(T X S) denote the number of points occurring in T x 5. Define the random variable 

A'{TxS)= [ Xt{r,xi,ni)dtdr 
JtxS 

Then (T x 5) is a doubly stochastic Poisson random variable with conditional probabil- 
ity distribution 

. , e-^"(^X'5) (A^ (T X 5))" 

Pr \N' (TxS) = n\A' {T x S)] = ^^-^ ^ 

n! 

Moreover, for disjoint TixSi and T2 x S2, conditioned on A' (7i x Si) and A-' {T2 x S2), the 
random variables [Ti x Si) and A^-' {T2 x S2) are independent for i = a,b and j = a, b. 
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Let {Cl,c!^,P) be the underlying probability space for the above stochastic model. De- 
fine as the cr-algebra generated by the space-time process i over [0,t). We define the 
counting process Nl as the number of points occurred during [0, t) over the entire surface of 
photodetector i regardless of their location, i.e., iVj = iV*([0,t) xM^). We say ttj and itj are 
admissible controls if ul is ^J-measurable and the solution to (jH) is well defined for i = a,b. 

Let T be an arbitrary positive constant. We define the objective functional 



J = E 



^ (a'^i/f exp(-p||C7tXti2) +a\^exp(-p||Ctx^f)) dt 



(9) 



where a* ^ 0, i = a,b. Evidently, J is a linear combination of the expected optical energy 
received by the stations during [0, T]. Our goal is to obtain admissible controls and 
that maximize J. 

4. The Control Problem 

We first outline some results in the estimation of the state vector xl due to Rhodes and 
Snyder [9]. Later, we will utilize these results to approach the control problem. Before 
moving forward, let us fix some notation. Let {ti^_i,tk] be the interval between two suc- 
cessive occurrence of the space-time process and be the location of k^^ occurring point. 
Assume that ht {r, ^j) is continuous in r and left continuous in t and ^t. Then the stochastic 
differential equation 

d^t= [ htir,Ct)N{dtxdr) 



is defined such that d^t = during (tfc-i,tfc] and S,t encounters a jump of ht,. {r^^Ctk) 
at t = tk- 

Consider the state-space model and its associated space-time observation with the 
rate process ([7]). Assume that we are given the increasing family of u-algebras and ul 
is =^^-measurable. Then, regardless of the nature of fj,], the posterior density p^i is 
Gaussian with mean and covariance matrix determined from the stochastic differential 
equations [9] 

dxi = Atx\dt + Btu[dt + / Ml [r - Ctx[) N' {dt x dr) (10) 

dS^ = At^dt + ^Ajdt + DtDjdt - Mid^dNi (11) 
with initial states Xq = Xq and Sq = Sq, where M^: is defined as 

Mi = j:ic^{Ct^c^+Rty' 

Moreover, the conditional covariance matrix obtained from ()lip is almost surely positive 
definite for t > 0, provided that Sq is positive definite. 

Note that the formulas (|10p and (llip not explicitly depend on {fil,t ^ 0}, however, the 
estimates xl and depend on {fil,t ^ 0} through the observation {dt x dr). This 
dependence can be explained by observing from (jlip that the occurrence of each event 
in the space-time process subtracts the positive definite matrix MjCjS^ from SJ, thus a 
larger //J leads to a smaller estimation error covariance by increasing the occurrence rate 
of the events. According to ([8]), a smaller pointing error ||Ctx^|| at station b results in a 



COOPERATIVE STOCHASTIC CONTROL FOR OPTICAL BEAM TRACKING 7 

larger /i" and, as a consequence, a closer estimation for , which in turn, leads to a smaller 
pointing error at station o. This explains the mechanism which couples the dynamics of 
the stations. 

We exploit the above results to prove theorem [T] below which determines an upper bound 
on J and establishes the conditions on and x\ under which the upper bound can be 
achieved. Before stating the theorem, we fix notation. Let S = [uij] denote a symmetric 
n X n matrix and / (E) be a scalar function of S. Assume that the partial derivatives of 
/ (S) with respect to elements of S exist. We denote by df (T,) /dT, a n x n symmetric 
matrix F (S) = [Fij (T,)] such that Fa = df/dau and Fij = (1/2) df /daij for i^j. Let 
gt (S", S^) be a function of n x n symmetric matrices and Ti^ with values in M. We 
define the linear operators {■} and Cf {■} as 

£1 {gt (S", S^) }=gt{St (S^) , S^) - gt (S^, S^) 
where St{-) is defined as 

St (S) = E - T.Cj{CtT.Cf + RtY^CtT. (13) 
With / being the 2x2 identity matrix, we define the 2x2 positive definite matrix Q\ as 

Q\ = {l + 2pCtT^,Cj)-"' (14) 

Also we define > as 

q\ = det {Q\) (15) 

Theorem 1. Fix sample paths and i/f, t G [0, T]. Let T,"" and be nxn symmetric ma- 
trices. Assume that gt (S",S''), t G [0,T] is the backward solution to the partial differential 
equation 

%(S«,S^)_ + {g,(E",S^)} ^ ,a' + dl{gt{T.-,T.^)] 

^t = + ■ 



Met (/ + 2pCtTPCf) ^det (/ + 2pCtT.^Cj) 

+ tr I ^^^^^ ( ^.S^ + S^Af + A ]\ (16) 

mi/i boundary condition gx (•, •) = 0. Then for fixed sample paths of Vt u^, the objective 
functional Q can be expressed as 

J = 50 (Sg,S[5) -E 



E 



^ i.,%^(a'^ + /:?{5i(Sf,S?)}) [l-e^Y>{-p\\Q\Ctx\f)}dt 
£ uU?{a' + L\ [at (S^ S,^)}) {l - exp(-p||Q»CiX»||2)}dt 



(17) 



Moreover, if YP" and are positive definite, >Cf {^^ (S", S**) } and Ct {gt {^°',^^)} are 
positive for t € [0, T), provided that Ct is full rank for any t £ [0, T). 



8 



ARASH KOMAEE 



Proof. See appendix El □ 

Corollary 1. For fixed sample paths and v\ and any choice of positive definite matri- 
ces Sg and Sq, J*— go {'^oi^o) ^■^ upper bound for J, i.e., J ^ J*, and equality holds 
if and only if Ctxf = for t € =^7" and Ctx\ = for t S almost everywhere. Here 2^°" 
and are defined for fixed sample paths and as = {t | f^* 7^ 0, t G [0)^)} CLf^d 

^^ = {t|z.f y^o, t G [o,r)}. 

Proof. The second statement of theorem[T]in conjunction with positive definiteness of Sj, i = 
a,b results C] {gt > for i = a, 6 and t G [0, T). It follows that the integrands 

in the first and second integrals of (|17p are positive over and respectively, except 
when CtXf = for t G and Cixf = for t G in which they are equal to zero. This 
leads to J — J* ^0 with equality when the integrals vanish. The last condition holds if and 
only if CtXf = for t G and Ctxf = for t G almost everywhere. □ 

Corollary 2. For nonnegative stochastic processes and with piecewise continuous 
sample paths and nonzero expectations, the objective functional ^ achieves its maximum 
if and only if 

Ctxt = Ctx\ = (18) 

for t G [0, T) almost everywhere. 

Proof. If (jlSp holds, corollary [1] implies that for any sample path of (ff,fj) the objective 
functional J associated to that sample path meets its upper bound. This suggests that (fT8]l 
is a sufficient condition for J to achieve its maximum. To show that (llSp is a necessary 
condition, assume that for some interval J' C [0,T), Ctx^ 7^ or Ctx\ 7^ 0. Because 
and have nonzero expectations, with nonzero probability some of their sample paths 
are positive over ^. Then, corollary [T] implies that for those sample paths the condition 
for achieving the maximum is not satisfied. Therefore, the objective functional J cannot 
achieve its maximum. □ 

Remark 1. According to (llOp and (jlip . and x\ do not explicitly depend on and v^. 
This suggests that the optimal control law not explicitly depends on and v^. 

Remark 2. The condition which leads to the upper bound J* does not depend on a*^ 
and . In particular, the condition is same for (a",a^) = (1,0) and {a^^a^) = (0,1). This 
means that if Ctx^ = Ctx\ = holds during t G [0,T), both stations receive the maximum 
possible optical energy. 

The following lemma proposes a control law which leads to the condition Ctx^ = Ctx\ = 0. 

Lemma 1. Consider the stochastic dynamical system (llOp and assume that CqXq = 0. 
Let CtBf be nonsingular and Ct be differentiable for t ^ 0. Then under control 

uldt = - (CtBt)-^ I (^CtAt + a) xldt + CtMl j ^ rN' {dt x dr)| (19) 
we have Ctx\ = for any t ^ 0. 
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Proof. We verify the validity of the lemma by putting 

uidt = - (CtBt)-^ I (CtAt + Ct^ x\dt + j CtMl (r - Ctx^ N' {dt x dr) | (20) 

into (fTO|) and left multiplying both sides by Ct- The resulting equation will be Ctdx\ = 
-Ctx\dt, which yields d {Ctx\) = 0. Then we argue that Ctx\ = CqXq = for t ^ 0. 
Finally, we put Ctx\ = into ([20]) to obtain □ 



5. Conclusion 

Cooperative optical beam tracking, a scheme for maintaining alignment in a free-space 
optical link, has been addressed. A stochastic model has been developed which captures 
three sources of randomness: relative motion of stations, characteristic of photodetectors, 
and fluctuation of optical power caused by optical fade and information-bearing signals 
modulating the optical beams. An optimal control law has been proposed which maximizes 
the expected optical energy received by stations of the link. It has been shown that under 
moderate assumptions, the control law does not depend on the characteristic of the optical 
fade or information-bearing signals modulating the optical beams. 



Appendix A. Proof of Theorem [T] 

We prove the first statement of the theorem through the following four steps. 
Step I: Recalling that p^i(x|=^|) is Gaussian with mean x\ and covariance SJ, it is straight- 
forward to show that 

E [e^v[-p\\Ctx\f)] = E [E [exp(-p||Cixjf ) |^,^]] 
= Y.[qle^v{-p\\Q\Ctx\f)] 

where Q\ and ql are defined by ()14p and (jlSp . respectively. Using the above equation we 
rewrite ([9]) as 



J=E 
-E 
-E 



d^v'ii{\ - exp [-p\\Q\Ctx\f)}dt 



f a\\qt{\ - exp (-p||Qf C^f f f) } 
./o 



(21) 



Step II: Let (i, j) = (a, 6), (6, a). For any t ^ and for any small positive e, ^NI=NIj^^ — Nl 
is a Poisson random variable conditioned on the rate 



Al = j uie^p{-p\\Crxif)dT 
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Thus, using the law of total probability we can write 



PrMM = 1 



= E 



Pr{AiV,^ = l!A^} 



where p\ ^ is defined as 



Pt,e 



t+e 



viqi^M-p\\QiCrxi\(')dT 



(22) 



(23) 



In a similar manner, we can show that 



Pr {AiV; = 0|=^^} = f - + O (e2) 
PrjAiV; ^ 2\ml^ = O (e^) 



(24) 



Step III: Let / (E) be a scalar function of nxn symmetric matrix E and assume / is 
differentiable with respect to E. We can write 



E [/ (Ej+J] = E [e [/ 

CO 



E 



E 



J;e[/ (Ey |AiV,^ = fc,^^' 



A;=0 
1 



Pr MiV? = k\ 



Pr <! ATV; = k\ 



n} 



k=0 



+ O {e') (25) 



Also we have 



E 



f{j:U,)\ANi = k,, 



E 



t+e) 



\T.l/^Ni = k,m 



E[/(Ej+,)|Ej,AiV; = A;] 



Inserting the last equation into (|25p . after some manipulation we find 



E[/(Ej+,)]=E 



1 

J^E [/ (Ej+J |Ej, A7V/ = A;] Pr {AiV,^ = k\m{] 



L k=0 



+ O (e^) (26) 



Conditioned on E^ and AA'^* = 0, 1, the stochastic differential equation (jlip can be solved 



for E^^^. This solution leads to 



E [/ (Ej+,) |Ej, AA^; = 0] = / (Ej + e^tE^ + eEj^f + eDtDj + O (e^)) 
E [/ (Ej+J |Ej, AA^; = 1] = / (Ej - MlCtT.\ + O (e)) 
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We linearize the above equations with respect to e and plug the linearized equations together 
with (I22l) and (ED into (IMl) to obtain 



E [/ = E [/ (S^) + 6 tr { {df (Sj) /asj) {At^ + ^Af + DtOf) } 

+ plJf m - MiCtn) - f + O (e^) 



(27) 

Let gt (£"^,5]^) be a scalar function of nxn symmetric matrices S° and and assume 
its partial derivatives with respect to t, and E'' exist. Then we can write 



E 



Applying (127p to gt and dgt/dt first for i = a and then for i = b, we find 



E 



E 



gt{J:i,^1)+edgt{^1,^1)/dt 



+ e tr { (a<7i (S?, S,^) /dEl) (^^E? + E,M^ + D^Df) } 
+ etr { (a<7i(S?, S,^)/aE,^) (a,E,^ + S^vl?^ 



In this equation, the term involving p^^ can be simplified as 



+ 0(e' 



(28) 



E 



pl (gt{^^-M,^Ct^1,^1)-gt{^l^ 



E 



E 



^ V"9rexp(-p||Qtaxtf)dT ■ CI {gt (Ef, E,^)} 
eu?q1e^p{-p\\QlCtx'tf)C1 {gt (E^E?)}! + O (e^) 



Here, the first equation is obtained from (|12p and ()23p and the second one is concluded from 
the fact that with probability 1 — O (e) the integrand in the first equation is continuous. 
Applying this result to (j28p and rearranging the equation we obtain 



E 



b 



E 



gt(E?,E?) -er* 



eE 



+ tr 



+ tr 



det (/ + 2paE^Cf ) 



det (/ + 2pCtT,^C^) 



dgti^l^ a «" + ^"{gt(E^E,^)} , , a^ + 4{g^ (S?,E^)} 

9i ' - - 

' %(S?,Eg) 
9Ef 

' %(S?,E^) 
5E? 



(29) 
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where is defined as 

r* = vt^^LI {at (Sf, S,^)} {1 - exp {-p\\Q\Ctx\f)] 

+ v\q^L\ {gt (Sf, S^) } {1 - exp {-p\\QlCtxtf) } (30) 

Assuming that 5^ (•, •) is the backward solution to (|16p with boundary condition (7^ (•,•)= 0, 
the right hand side of (j29p is identically zero, which yields to 



E 



E 



5t(S?,S?)-er< 



(31) 



Step IV: Let us denote the first term in the right hand side of (j2ip by J. We partition the 
interval [0, T) into K subintervals [i/o ^fc+i); A; = 0, 1, . . . , K — 1, where to = 0, = T, and 
tfc+i — tfc = Cfc > 0. Then, J can be approximated by 

K-\ 



k=0 



+ E 



Note that gt^^ {'i ')= Bt 0, thus the last term does not affect the sum and is arbitrarily 

added to the right hand side. The above equation can be rewritten as 

K~2 



k=0 



Since tx = tx-i + ex-i, we can apply ([3X1) to the second term of the right hand side of 
this equation to obtain 

K-2 
fc=0 

+ '^[9tKAn^-v^\^-S\- ^K-i^ [r*K-i] +o(4-i) 

Continuing this procedure for /c = — 2, X — 3, . . . , 0, we obtain 

K-l K-l 



9to (^to'^io 



^ e.E [r,J +Y,0 {el) 



k=0 



k=0 



Now we let -ftT — > 00 and maxe^ ^ to obtain 

J = hm Jk = go (Sg, S^) - E [ r Ttdt 

max efc^O 

Finally, we put this equation into ()2ip to obtain ()17p . 

In order to prove the second statement of the theorem, we need the following preliminar- 
ies: 

P-1) In the context of this proof, we say /{■)'■ M"^" — > M is strictly decreasing if for any 
symmetric positive definite matrices S and A, we have / (S + A) < /(S). Also 
we say / (•) is m-positive if for any symmetric positive definite matrix S we have 
/(S)>0. 
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P-2) If /i (•) and /2 (■) are strictly decreasing and m-positive, /i (•) + /2 (•) and /i (•) /2 (•) 

are strictly decreasing and m-positive as well. 
P-3) If / (•) is strictly decreasing, for any t in which Ct is full rank and any positive 

definite E we have / (S) < / (St (5])), where St (•) is defined by (fT3]) . 

Proof. Applying the matrix inversion lemma to (jl3p . it is easy to verify that St (S) 
is positive definite for any positive definite S. Also we know from ()13p that when Ct 
is full rank, A = T, — St (S) is a positive definite matrix. Because / (•) is assumed 
to be strictly decreasing, we can write / (S) = / {St (S) + A) < / (St (S)). □ 

P-4) If /(•) is strictly decreasing and m-positive, for any fixed t, f{St{-)) is strictly 
decreasing and m-positive. 

Proof. It is easy to verify that 

- (S + A)"^ = (E + SA-^S)"^ (32) 

holds for any invertible matrices S and A. Let S and A be positive definite matrices. 
Then ([32]) implies that A = S-^ - (S + A)"^ is a positive definite matrix. Using 
the matrix inversion lemma and replacing with (S -|- A)~ -|- A, we can write 

St (S + A) - St (S) 

= ((S + A)-V ClR^'CtY' - (S-i + C^Rt'Cty' 

= ((E + A)-i + C^R^CtY' - (((S + A)-^ + C^Rt'Ct) + a)"' 

Applying identity ([32]) to the last equation, we find that St (S + A) — St (E) is posi- 
tive definite. Then, because St (E) is positive definite and / (•) is strictly decreasing, 
we have 

/ {St (S + A)) = / {St (S) + {St (S + A) - St (S)}) < / {St (S)) 

which means / {St (•)) is strictly decreasing. Moreover, because / (•) is m-positive 
and St (S) is positive definite, / {St (•)) is m-positive. □ 

P-5) For any fixed t in which Ct is full rank, ht (S) defined as 

ht (S) = , ^ = 

^det {l + 2pCt^Cf) 

is strictly decreasing and m-positive. 

Proof. For positive definite T, and A we can write 

ht (S) / det (J + 2pCt^Cj + 2pCtAcf) 

ht (S + A) ~ Y det (/ + 2pCt^Cl) 

= Vdet (/-F2/9A*) 

where A* is defined as 

A* = ((/ + 2paSCn~'^' Ct) A ((/ + 2paSCn''^' Ctf 
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Because A is positive definite and Ct is full rank, A* is positive definite. This implies 
that det (/ + 2pA*) > 1, which leads to ht (S + A) < ht (S). □ 

P-6) Let be a scalar function of nxn matrices and S^. Assume that the 

function is strictly decreasing and m-positive in both and S^. For e > define 
the linear operator /Cj as 

where 

(E) = E + e (AE + EAf + DtDf) 

Then, for any symmetric positive definite matrices E", E'', A", and A'', there exists 
C = C (S", E^ A'^, A^) > such that for any ^ e < C we have 

/C,7t(S'^ + A^E^) </C,7t(S^E^) 

/C7i(E^E^ + A'') </C7t(E^E'') 

/C7t(E^E'') >0 

Therefore, as e — > 0"*", the above condition is satisfied for any choice of E"^, E**, 
A'*, and A*. This means that if e lays in a neighborhood of 0, however small, 
/C|/t (E", E^) is strictly decreasing and m-positive in both E*^ and E^. 

We claim that gt[T,"' , the solution of (fT6]) with boundary condition gxi-,-) = 0, is 
strictly decreasing in both E" and E^ for any t € [0, T), provided that Ct is full rank for t G 
[0, T). Once the claim is proven, we apply (P-3) to (fT2]) in order to show C} {gt (E", E'') } > 
0, i = a,b for any positive definite matrices E*^ and E'' and t G [0,T). 

To prove our claim, for any ^ t < T, we partition the interval [t, T) into K subintervals 
[tk+i,tk), k = 0,1, . . . ,K - 1, where tx = t, to = T, and tk - tfc+i = Cfc > 0. It is 
straightforward to discretise the partial differential equation (|16p over this partition to 
obtain the recursive equation 

9t,^. (S^ S^) = 6, [a-u^ht, (E^) + a%h, (E'^)) 

+ 6fc (E'')5t, {St, , S^) + ^^ht, (E'^)5i, (E^ St, (E^))) 

+ /C,>,(E^E^)+0(ei) (33) 

Starting from gtQ{-,-) = and using this recursive equation for k = 0,1,2, ... K — 1, 
we can determine gtj^ ('r)- Then by letting — > oo such that maxe^ — > 0, we have 
9tK {■,■) ^9t {■,■)■ 

We prove by induction that as K ^ oo and maxe^ 0, for k = 1,2, K , gt, (•, •) is 
strictly decreasing and m-positive in both E" and E^. From (P-2) and (P-5) we find out 
that 

gt, (E^ E^) = 60 (aV,>io (E^) + (E'^)) 

is strictly decreasing and m-positive. Now we show that if gt, (E", E^) is strictly decreasing 
and m-positive, gt,^i (E°, E^) is strictly decreasing and m-positive as well. For this purpose, 
we use (P-2, P-5) and (P-2, P-4, P-5) respectively to show that the first and second terms 
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on the right hand side of (|33]) are strictly decreasing and m-positive. Also as 0"*", 
(P-6) implies that the third term on the right hand side of (j33p is strictly decreasing and 
m-positive. Because all three terms on the right hand side of (I33p are strictly decreasing and 
m-positive, we conclude from (P-2) that gt^.^^ (S", S^) is strictly decreasing and m-positive. 
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